A distributed application environment typically consist of one or more client systems having parts of the distributed application, e.g. GUI components, processing components (client application), server systems system having the remaining parts of the distributed application (server application or server components), and a third tier server system which exchanges data between the client system and the server systems, e.g. provides a repository for the application data. For example SAP R/3 (trademark of SAP) is a typical example for a distributed application where the application is divided in a client part, e.g. running on a workstation like IBM Thinkpad, and a number of server parts, e.g. running on a IBM zSeries mainframe system.
The communication between the systems of the distributed application environment can take place in a secure or insecure communication environment. For example, a secure communication is normally given in the Intranet, and an insecure communication is given in the Internet. Distributed applications—particularly of those which handle sensitive information, such as operating system configuration data—demand that data interchange between the distributed components and with other applications such as databases should be protected by means of an established security mechanism.
A common way to protect the communication between client and server system is by establishing a secure session, where all communication is encrypted using a symmetrical key. This symmetrical key is exchanged between the partners using the public key cryptography. This exchange is typically combined with authentication of the partners.
The most prominent exponent of such security mechanisms is SSL (Secure Socket Layer) that represents “the most commonly-used protocol for managing the security of a message transmission on the Internet”. Amongst other things, SSL uses so called ‘server certificates’ to check and assure the identity of the server part in a connection. The certificate is used to distribute the public key. In addition it contains owner's name, certificate name, expiration date, public key of the signer of the certificate as well as a digital signature being generated by the owner's private key applied to the certificate. The checking, acceptance and administration of such certificates in each component of a distributed application may be a problem if no direct user-interaction is possible, which is particularly the case in server components.
By using that protocol the user benefits from a high level of protection with few effort. To establish a secure connection, the security protocol exchanges client or server certificates during the connection handshake. A definition and the message flow of SSL handshakes for example can be found in RFC 2246. Certificates identify and authenticate the participating parties.
The recipient can verify a certificate in two different ways. The first way, if the certificate has been signed by another certificate and that the certificate has been trusted before by the recipient, the new certificate can be trusted automatically. Already trusted certificates are kept in a database for the verification process in following connection handshakes. The second way, if the recipient has no certificate in its database which confirms the incoming peer certificate as trustworthy, the user must decide whether to accept or reject the certificate. This can be done by comparing the results of hash functions applied to certificate information with securely published values. If the user chooses to reject the certificate, the connection will not be established.